Internet Info 1994 March/Internet Info CD-ROM (Walnut Creek) (March 1994).iso/networking/news/nntp/inn/faq-inn-1

Text File | 1993-08-23 | 26.4 KB | 724 lines

Last Changed: $Id: FAQ-inn.1,v 1.41 1993/08/12 19:55:57 tal Exp $

                          Table Of Contents:
                          ------------------

QUESTIONS FROM PEOPLE THAT DON'T (YET) RUN INN:
    Where can I get it?
    What is INN?
    What machines does it run on?
    Can I run C News with INN?
    Can I run NNTP with INN?
    Can I run the reference implementation (NNTP1.5) with INN?
    Can I run INN on my UUCP-only machine?
    Suppose I have a 286 machine?
    Does INN implement trn's XTHREADS, etc?
    Does INN do UUCP batching like C News?
COMMON DEBUGGING/SETUP QUESTIONS:
    Help!?  How do I configure this beast
    Why does innd just exit right away with no message?
    I'm getting news but postings aren't going out.
    How come my host name comes out twice in the Path line?
    Why does my innd often die with the message "Can't sync history,
    7-bit encoded batches are not correctly processed. Why is this?
SPECIFIC OPERATING SYSTEM BUILD ADVICE:
    SunOS 4.1.2
    Ultrix tip 1 (mmap)
    Ultrix tip 2 (syslog)
    HP-UX
    SVR4, Solaris 2.x, and SCO ODT 3.0
OPERATIONAL QUESTIONS:
    Safe way to edit the "active" file?
    Expire had problems last night, and while I fixed the problem
        it still won't run.
    Why am I getting alt.sex.pictures even though I
        have "ME:!alt.sex.pictures" in my newsfeeds file?
    Why doesn't this newsfeeds entry do what I want?
    Why am I forwarded cancel messages for articles in comp.foo
        when I explicitly have !comp.foo in the newsfeeds entry?
    What's the best way to upgrade to a new version of INN?
ADVANCED QUESTIONS:
    What do I do if /var/spool/news is set up over many partitions?
    How do I talk to innd from C or Perl?
    How do I set up a delayed IHAVE/SENDME over NNTP?


(The FAQ was written by Rich $alz <[e-mail]> and is now
maintained by Tom Limoncelli <[e-mail]>.)


======================================================================

ADVANCED QUESTIONS:
            QUESTIONS FROM PEOPLE THAT DON'T (YET) RUN INN

======================================================================


------------------------------

Subject:  Where can I get it?

The official archive site is ftp.uu.net in the directory
networking/news/nntp/inn.  Archie current lists over 30 archive sites;
three other international sites are grasp.insa-lyon.fr in
pub/unix/news/inn, munnari.oz.au in pub/news/inn, and src.doc.ic.ac.uk
in computing/usenet/software/transport

This part of the FAQ can be gotten by sending email to
[e-mail] with "sendme faq-inn-1" in the body of the
message.  (the Subject: of the message will be ignored.) (Substitute
faq-inn-2 or faq-inn-3 to get those parts).

For example:

% mail [e-mail]
Subject: ignored
sendme faq-inn-1
sendme faq-inn-2
sendme faq-inn-3
^D

Or

% echo sendme faq-inn-1 | mail [e-mail]
% echo sendme faq-inn-2 | mail [e-mail]
% echo sendme faq-inn-3 | mail [e-mail]


------------------------------

Subject:  What is INN?

InterNetNews is a complete Usenet system.  The cornerstone of the package
is innd, an NNTP server that multiplexes all I/O.  Think of it as an nntpd
merged with the B News inews, or as a C News relaynews that reads multiple
NNTP streams.  Newsreading is handled by a separate server, nnrpd, that is
spawned for each client.  Both innd and nnrpd have some slight variances
from the NNTP protocol (although in normal use you will never notice); see
the manpages.  INN separates hosts that feed you news from those that have
users reading news.  If you need to support a mixed environment you will have
to do some extra work; the installation manual gives some hints.


------------------------------

Subject:  What machines does it run on?

If you have socket() and select() then INN will probably run on your
machine.  In addition to the common platforms found around the Internet
(Sun and Ultrix, for example), INN runs on AIX, A/UX, NeXT, and a host of
others.


------------------------------

Subject:  Can I run C News with INN?

No.  INN handles all article reception, filing, forwarding, and
expiration.  You will most likely get a corrupted database if you try to
run INN with any other news system.  For testing, you can probably shut
down your old system, bring up INN, and then reverse the process.  (INN
uses the C News history file and DBZ database, so if you don't run C News
you will have to do some fiddling around with those files.)


------------------------------

Subject:  Can I run NNTP with INN?

There's a confusion here.  NNTP is a protocol, defined in RFC 977.  There
is also an implementation of the protocol, NNTP1.5, that many people call
NNTP.  When there was only one implementation of the protocol, that was
okay, but now that there are other implementations (for example, INN) it
is getting confusing.  It would be as if "sendmail" were named "smtp."
Please try to be clear -- do you mean the NNTP protocol, or the NNTP
reference implementation currently maintained by Stan Barber?


------------------------------

Subject:  Can I run the reference implementation (NNTP1.5) with INN?

The quick answer is no.  INN listens on the NNTP port and handles all
incoming traffic.  It receives articles, files them, and arranges for them
to be forwarded to your peers.  If a site connects that is not listed as a
peer (e.g., a local workstation that does newsreading) then the INN server
hands the connection off to another program.  By default, this is nnrpd,
which implements the NNTP protocol for newsreaders (for example, it
includes the POST command but not the IHAVE command).  You can run the
reference implementation server instead of nnrpd if you want.  Doing this
can be useful if you have clients that want to do both reading and article
transfer.


------------------------------

Subject:  Can I run INN on my UUCP-only machine?

Sure.  While not designed for this, several people are running INN on
machines that do not have IP-connectivity (such as UUCP-only hosts) and
are quite happy with it.  You might want to give it a try, especially if
you think you will be joining the Internet some day.


------------------------------

Subject:  Suppose I have a 286 machine?

Won't work.  INN is designed to be a memory hog; a server that has been up
for a few days while will have a working set size of a few to several
megabytes, although not all of it will be resident.  For example, the
server keeps the active file and list of who gets what in memory, as well
as all articles that it is receiving.  Unless you can do things like
"malloc(64 * 1024)" without pain, INN won't work on your machine.


------------------------------

Subject:  Does INN implement trn's XTHREADS, tin's commands, etc?

The XTHREAD command has code but it is not supported; look at
$inn/nnrpd/nnrpd.h.  This code will probably vanish after 1.4.  Tin
commands are not supported.  Instead, INN supports Geoff Collyer's news
overview database, nov (world.std.com, src/news/nov.dist.tar.Z).  It
does not include the client package; it just updates and maintains the
overview databases.  For info on how to configure INN to use NOV, read
Part 3/3 of this FAQ.  (Subject: Cookbook example setting up NOV
("overchan").)


------------------------------

Subject:  Does INN do UUCP batching like C News?

Not as part of the standard distribution.  The batching system right
now is better than B News, but Rich has said he will be working on
improving that part of INN in a future release.  Christophe Wolfhugel
<[e-mail]> has written a package that is
very much like the C News batching system, however.  You can find it on
grasp.insa-lyon.fr in the pub/unix/news/inn/contrib directory.

Version 3 of Christophe's package includes a shell version and a Perl
version.  Version 4, not yet planned will only be in Perl. The
configuration file has evolved from older releases in order to support
new features like minimum batching".


======================================================================

                   COMMON DEBUGGING/SETUP QUESTIONS

======================================================================


------------------------------

Subject:  Help!?  How do I configure this beast

Tom Limoncelli wrote a very good tutorial document.  It is not part of
the INN release, but many FTP sites keep a copy in the same directory.
Look for a file named inn-tutorial.  It is part 2 of this FAQ.


------------------------------

Subject:  Why does innd just exit right away with no message?

First, fix your syslog: innd always logs a message before it exits.  (The
INN distribution includes a version of the current UCB syslog, along with
instructions on how to install it.  Ultrix systems might want to look at
the syslog that is available on gatekeeper.dec.com) Second, the most
common cause of this is that you do not have a history file (or no history
database).  You will see a message like this:
    ME cant dbminit /usr/local/news/history No such file or directory
This means that you do not have a history database.  You might want to run
the BUILD script in your INN source tree or read about makehistory in
doc/news-recovery.8; if you do the latter, make sure to rename the
database files.  This FAQ covers general questions about INN and questions
about how to compile it.  For information on configuration and debugging
one's configuration, see the "INN Configuration FAQ".


------------------------------

Subject:  I'm getting news but postings aren't going out.

You might find it helpful to read "Appendix IV:  First-time Usenet or NNTP
Installation" in the Install manual.  In order for postings to go out, you
must have a newsfeeds entry that names a site to receive them.  In the
standard case you want to record the article filename and Message-ID in a
batchfile for the site:
    myfeed/myfeed.dom.main:*,!foo.*\
    Tf,Wnm:
You now want to run send-nntp or nntpsend from cron so that the file is
flushed and that innxmit is called to read the file and send the articles
out.  See the manual pages.


------------------------------

Subject:  How come my host name comes out twice in the Path line?

The INN server puts its name in the Path line of every article that it
receives.  Obviously, it has to do this.  The default configuration has
inews put the local host in the Path header.  If nobody posts on the
server and you use fully-qualified domain names on your workstations,
then everything works the right way.  (If `hostname` doesn't give an
FQDN on your machine, you can work-around this by setting the "domain"
value in inn.conf; remember that innd never re-reads inn.conf.  You
must "ctlinnd shutdown x" and then re-start the server).  Many people
don't want the client machines to put their name in the Path header.
To do this, set INEWS_PATH to DONT.  Finally, let me say that it is
probably a mistake to have a "pathhost" line on any machine other than
your server if you set INEWS_PATH to DO.  If you doubt this, please
trace the article flow for yourself.  If you are curious about the
effect of INEWS_PATH, read the nroff source -- not the formatted
output -- of doc/inews.1


------------------------------

Subject:  Why does my innd often die with the message "Can't sync history,
      interrupted system call"

Are you running SunOS?  If so, the answer is that Sun broke the write
system call but a patch is available.  Any write could fail in this way,
it is just more likely to happen when writing large files and in-core DBZ
writes the history file out in one chunk.  See the "Known Problems"
section of the installation manual.  To the best of my knowledge, nobody
has seen this problem on any other system.


------------------------------

Subject: How do I create all those directories in the newsspool?

Q:  For example, if you receive comp.sys.amiga.applications, do you
have to mkdir /var/spool/news/comp/sys/amiga/applications?

A:  Nope.  innd creates the directory for you the first time you
receive an article for that newsgroup.


------------------------------

Subject: 7-bit encoded batches are not correctly processed. Why is this?

Chris Schmidt <[e-mail]> replies:

The decode program that comes with INN up to version 1.3 is broken.
Because of that the last article in a 7bit encoded batch will not
correctly be decoded (the last characters are screwed up).  This is
fixed in INN 1.4.


======================================================================

                SPECIFIC OPERATING SYSTEM BUILD ADVICE

======================================================================


------------------------------

Subject: SunOS 4.1.2

SunOS 4.1.2 (but not 4.1.1 or 4.1.3) broke the write system call but a
patch is available.  Any write could fail "half way", it is just more
likely to happen when writing large files and in-core DBZ writes the
history file out in one chunk.  See the "Known Problems" section of the
installation manual.


------------------------------

Subject: Ultrix tip 1 (mmap)

Ultrix has a "mmap()" function, but it doesn't do the same thing as the
SunOS/BSD mmap() function.  Therefore, do not configure INN to use
mmap() on a Ultrix system.  INN wants to find a mmap() function
that is like the one on SunOS/BSD systems.


------------------------------

Subject: Ultrix tip 2 (syslog)

The syslog on Ultrix sucks rotten eggs and Digital refuses to fix it.
(source: everyone that uses Ultrix and has ever used other systems)

Luckily, you can replace it with the routine that comes with INN.
However, some people have had better luck installing the syslog that
can be had on gatekeeper.dec.com:/pub/DEC/jtkohl-syslog-complete.tar.Z
It still works with old clients but does new-style syslogging, too.
Works great for me so far.  (this information from:  [e-mail]
(Nelson Minar)).  The syslog that is shipped with INN works pretty well
but there have been some claims that some old clients don't like it.


------------------------------

Subject: HP-UX

Q. I am running inn on an HP machine. Inn won't start up automatically
:-( I can start it manually. There is no problem with news or inn once
it is started.

A.  Try adding a "sleep 10" to the bottom of /etc/rc.news, or in
/etc/rc, right after /etc/rc.news is invoked.  On some machines,
including HP, the shell started by "#!/bin/sh" when /etc/rc is executed
will exit before innd has disassociated itself from that shell.  This
causes innd to exit, sometimes without printing an error message.
(source: [e-mail] (Paul Joslin ))

This problem goes away if you set HAVE_SETSID to "DO".  Something to do
with Posix Session Leader concepts. Ick. (source:
[e-mail] (Steve Howie))


------------------------------

Subject: System V based Unixes (SVR4, Solaris 2.x, SCO ODT 3.0, A/UX, ...)

If you are running any non-BSD (i.e. System V based) Unix you MUST have
the following option set:

##  How should close-on-exec be done?  Pick IOCTL or FCNTL.
#### =()<CLX_STYLE              @<CLX_STYLE>@>()=
CLX_STYLE               FCNTL

This includes SVR4, Solaris 2.x, A/UX and SCO ODT 3.0.  This is clearly
stated in the Install.ms file and repeated here since so many people
post to news.software.nntp after messing it up.

If it isn't FCNTL, you'll get tons of overchan processes hanging
around.  (source: Philip Gladstone <[e-mail]>)

In SCO ODT 3.0 and MOST systems, innd will link and run if you use
IOCTL but eventually will stop answering incoming calls.

Don't be fooled.  Just because it compiles doesn't mean it's going to
work.

Under Solaris 2.x (SunOS 5.x) you must add the following at the
beginning of each file using gethostbyname():

#define gethostbyname __switch_gethostbyname


------------------------------

Subject: Mac running A/UX 3.0

Use the INN malloc.


======================================================================

                        OPERATIONAL QUESTIONS

======================================================================


------------------------------

Subject:  Safe way to edit the "active" file?

The following sequence is the shortest:

ctlinnd pause "edit active"
[do something to the active file]
ctlinnd reload active "edit active"
ctlinnd go "edit active"

Simple!  No need to "flush" after step one.


------------------------------

Subject:  Expire had problems last night, and while I fixed the problem,
      it still won't run.

When expire starts up it "reserves" the server so that nobody else can
pause or throttle it.  This prevents anyone else from coming in and
modifying the history database.  If expire bails out because of a bad
error (e.g., your expire.ctl has syntax errors) it leaves the server
reserved so that no maintenance will be done until a good expire run has
occurred.  To unblock the server, use the ctlinnd "reserve" command with
an empty string argument.


------------------------------

Subject:  Why am I getting alt.sex.pictures even though I have
      "ME:!alt.sex.pictures" in my newsfeeds file?

The active file is the definitive list of what newsgroups you receive.
INN's ME entry is different from C News and B News; please see
newsfeeds.5.  If you do not want to receive alt.sex.pictures, ask the
system(s) that send you news not to send it to you.  (You would have to do
that no matter what news system you are running.)


------------------------------

Subject:  Why doesn't this newsfeeds entry do what I want?
      "foo.com:alt,!alt.sex"

A newsfeeds entry is not a sys file entry.  Please read newsfeeds.5.  You
might also find the sys2nf program in the frontends directory useful, as
well as the inncheck Perl script that is found in the samples directory.
The INN Configuration FAQ has cook-book examples of the steps required
to install a NNTP feed, UUCP feed, and NNTP via nntplink feed.


------------------------------

Subject:  Why am I forwarding cancel messages for articles in comp.foo
      when I explicitly have !comp.foo in the newsfeeds entry?

Control messages can be explicitly forwarded, so a control message to
comp.foo is forwarded to sites that recieve either comp.foo or control.
Please see the "Control Messages" section of innd.8.  As that
documentation says, you probably want to put "!control" in the
subscription list for most of your newsfeeds.


------------------------------

Subject:  What's the best way to upgrade to a new version of INN?

First, you should read the README and the Install.ms (yes, read
them both... again).  The README includes a technique to update
a new config.data file to be like your old one:

      % cd config
      % make subst
      % cp config.dist config.data
      % ./subst -f {OLDFILE} config.data
where "{OLDFILE}" names your old config.data file.

Now edit the config.data to see if you want to change any of
the new settings that didn't exist in the old version's config.data
file.

Compile everything:

    % cd $INN
    % make world

When you feel you are ready to install the new files shut the old daemon:

    % ctlinnd shutdown 'upgrade in progress'
    [ kill innwatch by hand if you need to ]

Install the new files:

    % cd $INN
    % make update

Now update all your $INN/site files to be the same as they were for
your old software.  "cd $INN/site ; make diff-installed" will tell
you what's different between the files in /usr/lib/news and $INN/site.
If you only make changes in the $INN/site directory and use "make install"
to copy them into place you'll save your self a lot of trouble.
Read $INN/site/Makefile for more interesting things that "make" can
do.

When you feel you are ready to install the new $INN/site files:

    % cd $INN/site
    % make install

Re-start the system:

    % sh /usr/lib/news/etc/rc.news

If everything was done right you should be up and running.  Part 2 of
the FAQ gives tips on testing your configuration.


------------------------------

Subject:  Can I use gzip with INN?

(this was written with the help of Michael Brouwer <[e-mail]>)

There are three things that can be effected by using gzip:  Compression
of old logs, compressing batches to send out, and decompressing batches
that come in.

With INN 1.4 all you need to do is change two lines in config.data to
something like this:

COMPRESS                /usr/local/bin/gzip
DOTZ                    .gz

If you rebuild INN with these options set, all logs will be gzipped, and rnews
will use gzip to decompress news.

gzip will automaticly and transparently decompress UNIX Compress, SCO
UNIX Compress (I'm told it's 99% compatible with UNIX Compress), Pack,
and gzip.  Therefore, you can now receive batches compressed with any
of the above listed formats.  Let's refer to this as "your site is now
a universal decompresser".

It has been reported that if you hardlink gzip to be zcat, and make
sure that it is the zcat that INN uses, you can get the "universal
decompresser" without having to use gzip for your logs.  (Though, gzip
for your logs is a big win, so why make trouble for yourself?)

`send-uucp' will still use compress for outgoing batches, so the sites
you feed won't suddenly start getting data they don't understand.

Before you can send gzipped batches, you should make sure that the
sites that you feed have made the above changes so that they have the
"universal decompresser" too.

Edit send-uucp to use gzip instead of compress for certain hosts (see
example of using compress -b12 for the host esac in send-uucp),
outgoing batches will be gzipped.

If you use sendbatch, you will have to edit the file so that COMPRESS
is set to "gzip" and COMPFLAGS is set to "-9vc".


======================================================================

                          ADVANCED QUESTIONS

======================================================================


------------------------------

Subject:  What do I do if /var/spool/news is set up over many partitions?

First of all, you can do this by either mounting a filesystem at
/var/spool/news/comp (for example) or by mounting a filesystem anywhere
and making /var/spool/news/comp a symbolic link to the new partition.

Articles will be written as normal, but cross-posts have to be handled
specially now.  Usually INN handles crossposts by writing the article
to the first newssgroup, and then creating hard links to all the other
places where the article should appear.  Hard links do not take up
additional disk space (except making your directories longer).  Hard
links also have the advantage that the file data doesn't get deleted
until the last hard link is gone (and they can be deleted in any
order).  Therefore, you can expire each newsgroup at a different rate,
but the file data won't delete until it is expired from the last
newsgroup.

The problem is that two hard linked files must both be on the same
filesystem (partition).

When INN sees that it can not make a hard link (because an article is
cross-posted across two partitions) it will try to make a symbolic
link.  If your system can not do symbolic links, set HAVE_SYMLINKS to
DONT in your config.data file.  This will make INN write a second
(or third, etc.) copy of the file instead.  (NOTE: INN 1.4 doesn't
make the extra files.  This feature is planned for INN 1.5.)

Anyway, even though INN will automatically create symbolic links, you
have to give expire the "-l" flag so that it will know to modify its
behavior.  Suppose that a message is posted to rec.photo and
alt.cameras and suppose that rec.photo expires more quickly then the
alt group.  If this happens, then you will be left with a dangling
symlink.  The -l flag prevents this from happening.

To inform expire that your spool is split across multiple partitions:

In news.daily, change:
    EXPIREFLAGS="-v1"
to read
    EXPIREFLAGS="-v1 -l"

In expirerm, change:
    RMPROC="fastrm -e -u -s ${SPOOL}"
to read
    RMPROC="fastrm -e -s ${SPOOL}"

You also have to edit innwatch and innstat.  After that, you're done!

If you ever need to run "makehistory" you should pay attention to this
caveat in makehistory(8):

    Makehistory does not handle symbolic links.  If the news
    spool area is split across multiple partitions, the follow-
    ing commands should probably be run before the database is
    regenerated:
        cd /usr/spool/news
        find . -type l -print | xargs -t rm

However, if /var/spool/news/comp was a symlink to the partition
that comp is really on, you'll have to reinstall that link.  You
have avoid this by changing the above commands to:
        cd /usr/spool/news
        find . -type l -print | grep '[0-9][0-9]*' | xargs -t rm

------------------------------

Subject:  How do I talk to innd from C or Perl?

Rich Salz says:

If you are writing C, look at doc/inndcomm.3 and include/inndcomm.h;
they include all you need to do any ctlinnd command (in fact, ctlinnd
itself is little more than a call to the library).

Hacking up a Perl subroutine that spoke to innd's Unix-domain control
socket should be fairly straightforward but hasn't yet been written.


------------------------------

Subject:  How do I set up a delayed IHAVE/SENDME over NNTP?

Christophe Wolfhugel <[e-mail]> writes:

INN now allows to generate a timestamp entry in the batchfiles or to
the channels/exploders (Wt in newsfeeds) which can be used to allow for
example delayed ihave/sendme processing. INN's senders (like innxmit)
do not use that data yet.

[e-mail] has written a small patch for
nntplink 3.1.0 which supports this, the patch is available in his
anonymous ftp:
    grasp.insa-lyon.fr:pub/unix/news/nntp/nntplink/delayed-1.0.pch
The patch has been incorporated into nntplink 3.2, which has finally
been released.  The syntax that you would use in your newsfeeds
file would be:

    site:*:Tf,Wnmt:

and run this command now and then:

    nntplink -i batchfile -y 300 -b site news.site.fr

The delayed IHAVE/SENDME is expected to allow bandwidth savings in
situations where all sites use nntplink in following topology:

     Your site -- 64k -----------+-----------  Site 1
                                 |               |
                                 |              2mb
                                 |               |
                                 +------------ Site 2

   Site 1 and 2 are in the same metropolitan area, you feed them both.
   With the standard nntplink layout, you generally send all articles
   twice, which is a waste even if you're at 2 Meg/s link and even if
   Site 1 and 2 do nntplinks, you're faster.

   The delayed link would be used between your site and Site 2.  A 2 or
   3 minute delay allows Site 1 to feed Site 2 before you, and in case
   of a Site 1 outage the backup starts nearly immediately.

   Reasonnable delays are still kept as You -> 1 -> 2 should take less
   than one minute (or just 300 ms disk to disk if using nntplink -i ? :)).

Experiences seem to show that a 2 to 3 minutes delay is
a reasonable choice.

Chris